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United  States  General  Accounting  Office 
Washington,  DC  20548 


October  10,  2001 

The  Honorable  Bob  Schaffer 
House  of  Representatives 

Dear  Mr.  Schaffer: 

The  Office  of  Juvenile  Justice  and  Delinquency  Prevention  (OJJDP)  was 
established  to  help  states  and  communities  prevent  and  control  juvenile 
delinquency  and  improve  their  juvenile  justice  systems.1  OJJDP  provides 
funding  to  states,  territories,  localities,  and  private  organizations  through 
block  grants2  and  discretionary  funding.3  Between  September  1996  and 
September  2000,  the  number  of  active  OJJDP  discretionary  grants  has 
more  than  tripled,  from  240  to  790,  and  the  related  funding  for  active 
grants  almost  doubled  from  $286  million  to  $555  million. 

Monitoring  these  grant  activities  is  a  key  management  tool  to  help  ensure 
that  funds  are  being  properly  spent.  In  May  1996,  we  testified  that  the 
official  grant  files  for  discretionary  grants  generally  contained  plans  for 
agency  monitoring,  but  little  evidence  that  it  occurred.4 

Your  original  request  asked  about  various  aspects  of  OJJDP’s  grant 
management.  We  will  be  responding  to  that  request  with  two  reports.  This, 
the  first,  addresses  how  well  OJJDP  monitors  its  grants  to  ensure  that 
grantees  properly  implement  the  programs  that  the  grants  support.  The 
second  report  will  discuss  OJJDP  grantee  reporting  requirements,  the 


IrThe  Juvenile  Justice  and  Delinquency  Prevention  Act  of  1974,  P.L.  93-415,  as  amended,  42 
U.S.C.  5601,  et  seq.  It  is  one  of  five  components  of  the  Department  of  Justice’s  Office  of 
Justice  Programs  (OJP);  the  other  four  components  are  the  National  Institute  of  Justice, 
Bureau  of  Justice  Statistics,  Bureau  of  Justice  Assistance,  and  Office  for  Victims  of  Crime. 

2Block  grants  are  awarded  to  the  states  to  provide  assistance  to  state  and  local  units  of 
government  for  programs  in  accordance  with  legislative  requirements. 

3Discretionary  grants  are  awarded  to  states,  units  of  local  government,  or  private 
organizations  at  the  discretion  of  the  awarding  agency.  Most  discretionary  awards  are 
competitive  in  nature  in  that  there  are  limited  funds  available  and  a  large  number  of 
potential  recipients. 

testimony  before  the  Subcommittee  on  Youth  Violence,  Committee  on  the  Judiciary,  U.S. 
Senate,  Juvenile  Justice:  Selected  Issues  Relating  to  OJJDP’s  Reauthorization 
(GAO/T-GGD-96- 103,  May  8, 1996). 
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number  of  juveniles  OJJDP  programs  have  directly  served,  and  our 
analysis  of  the  program  impact  evaluations  OJJDP  has  funded  of  its 
programs. 

Specifically,  we  reviewed  and  are  reporting  on  the  (1)  OJJDP  grant 
monitoring  process,  (2)  extent  to  which  OJJDP  documented  its 
discretionary  grant5  monitoring  activities,  and  (3)  process  it  uses  to 
oversee  those  activities.  In  appendix  I,  we  describe  OJJDP’s  award  process 
for  discretionary  grants. 

To  address  these  areas,  we  met  with  officials  from  OJJDP  and  OJP 
concerning  discretionary  grant  monitoring  activities,  including  OJJDP’s 
activities  to  oversee  its  monitoring.  We  reviewed  OJP  and  OJJDP’s  grant 
monitoring  requirements  and  selected  and  reviewed  a  representative 
sample  of  OJJDP  demonstration  discretionary  grants  that  were  active  in  all 
of  fiscal  years  1999  and/or  2000.  We  did  this  to  determine  whether  OJJDP 
officials  were  adhering  to  the  agency’s  discretionary  grant  monitoring 
requirements  and  how  the  results  of  its  monitoring  efforts  were 
documented  in  its  grant  files.  We  also  made  a  similar  review  of  a 
representative  sample  of  OJJDP  training  and  technical  assistance  grants.6 
To  determine  OJJDP’s  oversight  role  of  its  discretionary  grant  monitoring 
activities,  we  reviewed  information  such  as  performance  work  plans.  We 
also  discussed  with  OJJDP  officials  the  steps  it  has  taken  to  improve 
previously  reported  monitoring  problems. 

Our  analysis  did  not  address  the  quality  of  how  OJJDP  monitored  its 
grantees.  For  example,  we  reviewed  documentation  to  determine  whether 
OJJDP  made  required  site  visits,  but  we  could  not  review  the  quality  of  the 
site  visit  contact. 


Results  in  Brief 


OJJDP  has  specific  program  monitoring  and  documentation  requirements 
for  its  discretionary  grants.  These  monitoring  requirements  include  such 
activities  by  the  grant  manager  as  making  quarterly  telephone  calls,  on  and 
off-site  grant  monitoring  visits,  and  reviewing  interim  and  final  products. 

In  addition,  all  discretionary  grantees  are  required  to  submit  categorical 


includes  cooperative  agreements. 

6For  both  discretionary  and  training  and  technical  assistance  grants,  we  express  our 
confidence  in  the  precision  of  our  sample  results  as  a  95-percent  confidence  interval. 
Unless  otherwise  noted,  all  percentage  estimates  have  confidence  intervals  plus  or  minus 
10  percentage  points  or  less. 
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assistance  progress  reports  that  summarize  project  activities  and  quarterly 
financial  status  reports  for  review  by  grant  managers.  OJJDP  also  requires 
certain  documentation  at  the  time  of  grant  closeouts.  At  the  time  of  our 
review,  OJJDP  was  reviewing  its  monitoring  practices,  including  how  to 
improve  its  grant  records. 

Our  review  of  OJJDP’s  official  grant  files  and  grant  managers’  files  for  the 
most  recent  award  for  a  representative  sample  of  grants  active  in  all  of 
fiscal  years  1999  and/or  2000,  showed  that  OJJDP  was  not  consistently 
documenting  its  grant  monitoring  activities.  Our  current  observations  are 
similar  to  those  we  reported  in  May  1996  about  the  agency’s  lack  of 
documentation  of  its  monitoring  activities. 

The  grant  managers’  monitoring  plans  for  the  most  recent  awards  for  79  of 
89  demonstration  grants  and  34  of  45  training  and  technical  assistance 
grants  had  specific  requirements  for  telephone  contacts  (e.g.,  monthly  or 
quarterly).  There  was  no  documentation  showing  that  the  requirement  for 
telephone  contacts  was  met  in  96  percent  of  the  demonstration  grants  and 
in  any  of  the  training  and  technical  assistance  grants. 

The  grant  managers’  monitoring  plans  for  the  most  recent  awards  for  66 
demonstration  grants  and  20  training  and  technical  assistance  grants  had 
specific  requirements  for  on-site  visits.  There  was  no  documentation 
showing  that  the  requirement  for  site  visits  was  met  in  88  percent7  of  the 
demonstration  grants  and  90  percent8  of  the  training  and  technical 
assistance  grants. 

Progress  and  financial  reports  filed  by  the  grantee  are  required  for  the 
entire  grant  period.  In  56  percent  of  the  89  demonstration  grant  files  and 
80  percent9  of  the  45  training  and  technical  assistance  grants  files 
reviewed,  the  progress  reports  did  not  cover  the  entire  grant  period.  In  65 
percent  of  the  89  demonstration  grant  files  and  60  percent10  of  the  45 
training  and  technical  assistance  grant  files  reviewed,  the  financial  status 
reports  did  not  cover  the  entire  grant  period. 


7The  estimated  range  is  between  77  and  95  percent. 

*The  estimated  range  is  between  68  and  99  percent. 

°The  estimated  range  is  between  69  and  88  percent 

10The  estimated  range  is  between  48  and  71  percent 
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OJJDP  does  not  systematically  monitor  grant  managers’  compliance  with 
its  monitoring  requirements  or  guidance,  or  assess  the  effectiveness  of 
OJJDP  grant  monitoring  practices.  According  to  OJJDP  officials,  one 
method  of  overseeing  compliance  with  the  monitoring  requirements  is 
their  review  of  performance  work  plans.  However,  these  plans  referred, 
for  example,  to  monitoring  grants  to  ensure  that  the  projects  meet  their 
goals  and  milestones,  with  no  clear  expectations  regarding  monitoring 
requirements,  such  as  site  visits  or  telephone  monitoring,  or  regarding 
documentation  of  these  contacts  and  reviews  of  grantees  progress  and 
financial  reports. 

We  are  making  recommendations  regarding  OJJDP’s  need  to  improve  its 
documentation  of  discretionary  grant  monitoring  activities. 

The  Acting  Assistant  Attorney  General  provided  us  with  written  comments 
on  a  draft  of  this  report  by  letter  dated  September  19,  2001.  In  general,  she 
agreed  with  our  findings  and  recommendations.  However,  she  stated  that 
we  had  not  adequately  considered  the  fact  that  the  associated  cost  of 
monitoring  has  not  been  commensurate  with  OJJDP’s  available  budgetary 
resources.  She  also  commented  that  we  had  not  adequately  considered 
OJJDP’s  recent  efforts  in  a  major  OJP-wide  effort  to  improve  grant 
monitoring  practices  and  establish  uniform  policies. 

While  we  are  encouraged  by  OJJDP’s  stated  dedication  to  improving  its 
grant  monitoring  practices,  we  believe  the  results  of  our  review  are 
sufficient  to  conclude  that  OJJDP’s  discretionary  grant  monitoring  is  not 
effective.  We  do  recognize  in  the  report,  the  rapid  growth  in  discretionary 
grants  over  the  past  5  years.  However,  we  believe  that  monitoring  is  a 
critical  management  control  that  allows  OJJDP  to  know  that  federal 
dollars  have  been  appropriately  spent.  This  conclusion  is  supported  by  the 
Department  of  Justice’s  Office  of  the  Inspector  General  (OIG).  In  a 
December  2000,  report  to  the  Congress,11  the  Inspector  General  cited  grant 
management  as  one  of  the  10  most  serious  management  challenges  facing 
Justice.  The  Inspector  General  stated  that  Justice’s  multi-billion  dollar 
grant  programs  are  a  high  risk  for  fraud  given  the  amount  of  money 
involved  and  the  tens  of  thousands  of  grantees.  Among  other  things,  the 
Inspector  General  said  that  past  OIG  reviews  have  determined  that  many 
grantees  did  not  submit  the  required  program  monitoring  and  financial 
reports  and  that  program  officials’  on-site  monitoring  reviews  did  not 
consistently  address  all  grant  conditions.  The  full  text  of  the  Acting 


11 Management  Challenges  in  the  Department  of  Justice  (Dec.  1, 2000). 
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Assistant  Attorney  General’s  comments  and  our  evaluation  of  them  are 
presented  in  appendix  II  and  elsewhere  in  this  report,  as  appropriate. 


Background 


OJJDP,  within  the  U.S.  Department  of  Justice,  is  the  primary  federal 
agency  responsible  for  the  prevention  and  control  of  juvenile  delinquency 
in  the  United  States.  This  includes  preventing,  treating  and  controlling 
youth  gang  activity  and  violence  through  sponsored  research,  evaluation, 
and  demonstrating  new  approaches  for  communities  across  the  country. 
The  initiatives  are  carried  out  by  seven  components  within  OJJDP: 
Research  and  Program  Development  Division,  Training  and  Technical 
Assistance  Division,  Special  Emphasis  Division,  State  Relations  and 
Assistance  Division,  Information  Dissemination  Unit,  Concentration  of 
Federal  Efforts  Program,  and  Child  Protection  Division.  OJJDP  annual 
appropriations  have  more  than  tripled  from  $162  million  in  fiscal  year 
1996,  with  71  authorized  staff  positions,  to  $568  million  in  fiscal  year  2000, 
with  87  authorized  staff  positions. 

In  1996,  we  reviewed  the  operations  of  OJJDP.  We  found  that  official  grant 
files  for  discretionary  grants  generally  contained  monitoring  plans,  but 
little  evidence  that  monitoring  occurred.  We  reported  that  none  of  the 
grant  files  had  documentation  of  telephone  contacts,  site  visits,  or  product 
reviews.  Also,  no  quarterly  program  and  financial  reports  were  included  in 
1 1  of  the  78  files  for  which  projects  had  been  ongoing  for  at  least  2 
quarters  at  the  time  of  our  review,  and  one  or  more  reports  were  missing 
from  another  61  files.  In  addition,  only  6  of  the  78  files  had  program  and 
financial  reports  for  all  quarters  of  work  completed. 

In  commenting  on  our  testimony,  the  OJJDP  Deputy  Administrator  said 
that  the  heavy  workload  of  OJJDP  staff  may  have  resulted  in  a  lack  of 
monitoring  records  and  that,  as  a  result  of  the  our  1996  review,  OJJDP 
would  take  the  steps  necessary  to  improve  records. 


Scope  and 
Methodology 


To  address  the  first  objective,  to  report  on  OJJDP’s  grant  monitoring 
process,  we  reviewed  applicable  federal  laws  and  regulations  as  well  as 
OJP  and  OJJDP  policies  and  procedures.  We  also  interviewed  staff  in 
OJJDP  and  OJP’s  Office  of  the  Comptroller. 

To  accomplish  the  second  objective,  to  report  on  OJJDP’s  documentation 
of  its  discretionary  grant  monitoring  activities,  we  reviewed  probability 
samples  of  official  grant  files  and  grant  managers’  files  using  a  data 
collection  instrument  to  record  whether  required  monitoring  activity 
occurred— that  is,  that  the  progress  reports,  financial  reports,  and  other 


Page  5 


GAO-02-65  Better  Grant  Monitoring  Needed 


required  documents  were  included  in  the  files.  For  each  grant,  we  also 
reviewed  the  most  recent  award  covering  12  months  to  examine  specific 
monitoring  requirements  and  activities,  such  as  telephone  calls  and  site 
visits.  Specifically,  we  reviewed  a  random  sample  of  89  of  545  OJJDP 
demonstration  discretionary  grants,  and  45  of  83  training  and  technical 
assistance  discretionary  grants  that  were  active  in  all  of  fiscal  years  1999 
and/or  2000.  The  results  of  these  samples  are  representative  of  the 
populations  from  which  they  were  drawn.  We  express  our  confidence  in 
the  precision  of  our  sample  results  as  a  95-percent  confidence  interval. 
Unless  otherwise  noted,  all  percentage  estimates  have  confidence 
intervals  plus  or  minus  10  percentage  points  or  less. 

To  determine  the  timeliness  of  grantee  progress  reports  and  financial 
status  reports,  we  reviewed  all  such  reports  for  OJJDP  demonstration  and 
training  and  technical  assistance  discretionary  grants  from  the  initiation  of 
the  grant  through  the  date  of  our  review. 

For  our  review,  we  provided  a  list  of  the  selected  grants  to  OJJDP  and 
OJP’s  Comptroller.  Because  we  did  not  always  receive  the  grant  files  in  the 
order  requested,  we  reviewed  them  out  of  order.  When  files  were  missing 
or  lost,  we  substituted  replacement  files.  After  our  review  was  complete, 
some  of  the  missing  files  were  located.  We  did  not  review  the  missing  files 
but  used  the  substitute  files  that  were  a  part  of  our  random  sample.  Four 
grant  files  were  not  found  in  time  to  be  included  in  our  review.12  In 
addition,  when  only  one  of  the  two  files  (the  official  grant  file  or  the  grant 
managers’  file)  was  available,  we  used  the  available  file  to  complete  the 
data  collection  instrument.  We  have  no  reason  to  believe  that  the 
substitution  of  files  affected  the  validity  of  our  sample  because  they  were  a 
part  of  our  random  sample. 

To  determine  whether  OJJDP  grant  closeout  procedures  were 
implemented  in  accordance  with  OJP  policy,  we  reviewed  grant  files  in 
our  sample  whose  project  periods  expired  before  August  31,  2000.  For  our 
review,  we  focused  on  required  closeout  documentation,  such  as  final 
progress  or  performance  reports,  final  financial  status  reports,  grant 


12In  commenting  on  a  draft  of  this  report,  the  Acting  Assistant  Attorney  General  stated 
reasons  for  the  four  official  grant  files  not  being  accessible  during  our  review.  She  said  one 
file  was  not  provided  due  to  an  incorrect  grant  number  on  our  list  provided  to  the 
Comptroller’s  Control  Desk  when  the  official  grant  files  were  requested.  However,  the 
incorrect  grant  number  was  contained  in  the  list  of  grants  that  OJJDP  provided  to  us.  One 
of  the  remaining  grant  files  had  been  sent  to  storage.  OJP  staff  had  checked  out  a  third,  and 
the  last  was  missing.  This  last  file  is  being  reconstructed  according  to  the  Acting  Assistant 
Attorney  General. 
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checklist,  and  closeout  notification  and  grant  ac^justment  notice.  Our 
results  should  not  be  projected  to  the  universe  of  cases  in  our  sample. 

To  address  the  third  audit  objective,  to  report  on  the  processes  OJJDP 
uses  to  oversee  monitoring  activities,  we  examined  OJJDP’s  monitoring 
and  oversight  of  its  grant  managers’  compliance  with  applicable  laws, 
regulations,  and  procedures  for  monitoring.  We  obtained  information  from 
OJJDP  officials  at  headquarters  and  obtained  data  related  to  these 
activities,  such  as  performance  work  plans.  To  determine  the  requirements 
in  the  performance  work  plans,  OJJDP  officials  gave  us  a  plan  that  they 
considered  to  be  representative. 

Our  work  was  done  between  March  2001  and  September  2001  in 
accordance  with  generally  accepted  government  auditing  standards. 


Discretionary  Grant 
Monitoring  Process 


After  OJJDP  awards  discretionary  grants,  OJP  policies  require  OJJDP  to 
monitor  the  grants  and  related  activities  and  document  the  monitoring 
results  in  the  grant  managers’  program  files  and  OJP’s  Office  of  the 
Comptroller  official  grant  files.  The  monitoring  is  to  ensure 

compliance  with  relevant  statutes,  regulations,  policies,  and  guidelines; 
responsible  oversight  of  awarded  funds; 

implementation  of  required  programs,  goals,  objectives,  tasks,  products, 
time  lines,  and  schedules; 

identification  of  issues  and  problems  that  may  impede  grant 
implementation;  and 

adjustments  required  by  the  grantee  as  approved  by  OJJDP.13 

One  of  the  OJJDP  grant  managers’  main  responsibilities  is  project 
monitoring.  Each  grant  manager  prepares  a  monitoring  plan  as  part  of  a 
grant  manager’s  memorandum  recommending  initial  or  continuation 
funding.  The  level  of  monitoring  required  is  based  upon  the  stated 
monitoring  plan  in  the  grant  manager’s  memorandum.  The  memorandum 
includes 


•  an  overview  of  the  project; 

•  a  detailed  description  of  what  type  of  activities  the  grantee  plans  on 
implementing; 


I3OJP,  OJP  Handbook:  Policies  and  Procedures  for  the  Administration  of  OJP  Grants 
(OJP  HB  4500.2C,  Feb.  1992),  p.  93. 
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a  discussion  of  past  monitoring  activities  and  assessments,  if  the  grant  is  a 
multiyear  grant  that  is  awarded  yearly; 

an  identification  of  monitoring  activities  to  be  performed  for  the  current 
project  period;  and 

a  discussion  of  the  financial  justification  for  the  grant  funds  and  of  the 
cost-effectiveness  evaluation  of  the  application. 

Another  part  of  the  monitoring  plan  may  list  specific  activities  or 
milestones  to  be  monitored  by  the  grant  manager. 

The  1991  OJJDP  Monitoring  Desk  Book  contains  specific  monitoring 
procedures  that  grant  managers  used  for  monitoring  OJJDP’s  discretionary 
grants.  Monitoring  activities  documented  included  site  visit  reports  and 
quarterly  monitoring  telephone  calls  as  well  as  other  significant  telephone 
calls. 

Also,  according  to  the  OJP  1992  Handbook,  all  discretionary  grantees  are 
required  to  submit  categorical  assistance  progress  reports  that  summarize 
project  activities  and  quarterly  financial  status  reports  for  review  by  grant 
managers.  The  OJP  Handbook  also  stated  that  when  all  of  the  grant  funds 
are  spent,  all  grant  activities  are  completed,  and  all  grantee  and  grant 
managers’  administrative  requirements,  including  all  paperwork  and 
review  of  grant  files,  are  completed,  the  grant  managers  have  to  close  out 
the  grant.  (See  app.  I  for  a  detailed  discussion  of  the  grant  award 
processes.) 

The  OJP  Comptroller’s  Control  Desk  is  to  maintain  the  official  grant  files 
and  is  responsible  for  tracking  the  receipt  of  all  grant  documents.  The 
Control  Desk  receives  the  progress  reports,  files  the  original  in  the  official 
grant  file,  and  forwards  a  copy  to  the  cognizant  program  office.  The  grant 
manager  is  responsible  for  the  timely  acceptance,  review,  and  analysis  of 
progress  reports. 

The  Comptroller’s  Monitoring  Division  performs  an  in-house  review  of 
various  program  and  financial  documents  contained  in  the  official  grant 
files.  The  Office  of  the  Comptroller  also  performs  risk-based  on-site 
financial  reviews  of  grantee  organizations  to  monitor  administrative  and 
financial  capability.  The  Comptroller  applies  risk-based  criteria  to  a 
universe  of  grants  to  develop  a  sample  for  each  fiscal  year  monitoring 
plan.  The  risk-based  criteria  include  the  dollar  amount  of  the  grant,  new 
grantees,  new  grant  programs,  and  programs  with  known  problems.  The 
Comptroller  excludes  from  its  sample  any  grantee  organizations  that  were 
subject  to  its  other  on-site  financial  monitoring  or  to  audit  by  the  Office  of 
the  Inspector  General  during  the  last  2  years.  According  to  OJP  officials, 
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the  purpose  of  the  reviews  is  to  provide  as  much  of  a  comprehensive 
financial  review  as  possible  without  going  on  site. 14  Our  review  of  the 
results  of  the  Comptroller’s  in-house  financial  reviews  of  the  grants  in  our 
sample  disclosed  that  they  identified  a  number  of  problems  similar  to  our 
own  observations.  These  included  missing  program  and  financial 
documents  and  program  and  financial  monitoring  not  always  being 
documented  in  a  timely  manner.15 

In  commenting  on  a  draft  of  this  report,  the  Acting  Assistant  Attorney 
General  stated  that  there  was  no  mention  of  delinquent  documentation  of 
financial  monitoring  during  our  review,  nor  was  there  any  evidence  cited 
in  our  draft  report  to  support  the  contention  that  financial  monitoring  was 
not  documented  in  a  timely  manner.  However,  contrary  to  the  Acting 
Assistant  Attorney  General  comment,  our  review  found  that  there  was  no 
documentation  that  financial  reports  were  generally  reviewed  and  that 
many  of  the  reports  were  not  submitted  in  a  timely  manner.  Specifically, 
we  found  one  case  where  no  progress  or  financial  reports  had  been 
submitted  during  the  life  of  a  grant,  and  it  was  not  discovered  until  almost 
a  full  year  after  the  award  date  (see  p.  15). 


Outdated  Monitoring 
Policies  and  Procedures 
Were  Being  Used 


In  1991,  OJJDP  issued  a  Monitoring  Desk  Book,  which  incorporated 
material  from  OJP’s  1987  Handbook,  including  a  formula  for  determining 
the  monitoring  level  requirements  for  the  grant.  According  to  OJJDP 
officials,  this  formula  is  applied  to  determine  how  often  a  grantee  will  be 
visited  and  officially  contacted  for  monitoring  purposes.  This  formula 
applies  a  weight  to  issues  such  as  program  priority  (i.e.,  how  important  is 
the  grant  within  the  context  of  the  OJJDP’s  goals  and  objectives),  size  of 
grant,  grantee’s  need  for  assistance,  and  anticipated  difficulties  in 
implementing  the  approach.  The  grant  manager  assigns  a  weight  or  “score” 
to  each  of  these  criteria,  which  are  added  together  for  a  total  score.  This 


14In  October  2000,  the  Comptroller  eliminated  its  review  of  progress  reports  because 
Comptroller  officials  believed  that  it  was  a  duplication  of  efforts  between  the  program 
offices  and  the  Comptroller's  Office,  and  it  wanted  to  streamline  closure  of  outstanding 
reviews. 

15The  Monitoring  Division  made  939  in-house  financial  reviews  in  fiscal  year  1999,  of  which 
one  OJJDP  demonstration  grant  and  five  training  and  technical  assistance  grants  were 
included  in  our  review.  In  fiscal  year  2000,  the  division  made  1,248  financial  reviews,  of 
which  three  OJJDP  demonstration  grants  and  one  training  and  technical  assistance  grant 
were  included  in  our  review.  As  of  August  2001,  the  financial  reviews  included  eight  OJJDP 
demonstration  grants  and  three  training  and  technical  assistance  grants  that  were  included 
in  our  review. 
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score  assigns  the  grant  to  a  category  that  defines  the  specific  number  of 
on-site  and  off-site  visits  and  the  frequency  of  monitoring  calls. 

In  February  1992,  OJP  revised  its  1987  Handbook  and  eliminated  the  score 
requirement  to  determine  the  monitoring  level  of  a  project.  However, 
OJJDP  grant  managers  still  use  the  instructions  to  determine  the 
monitoring  level  of  a  project.  We  found  in  our  file  reviews  that  72  of  the  89 
demonstration  grant  files  and  14  of  the  45  training  and  technical  assistance 
grant  files  had  a  score  or  level  indicating  monitoring  requirements.  Using 
the  obsolete  monitoring  requirement  to  establish  grant  manager’s  grant 
monitoring  responsibilities  may  not  result  in  the  most  efficient  use  of  their 
time  or  in  the  needed  monitoring  to  ensure  grantee  compliance,  as  pointed 
out  by  the  Deputy  Administrator  in  May  1996.  He  said  that  site  visits  did 
not  occur  because  they  are  expensive  and  time-consuming.  He  added  that 
OJJDP  might  need  to  revise  some  procedures,  noting  that  a  site  visit  once 
every  2  years  and  some  interim  telephone  monitoring  may  be  more 
appropriate  than  annual  visits.  Current  OJP  policy  provides  flexibility  to 
determine  monitoring  needs  (e.g.,  site  visits  as  appropriate)  consistent 
with  resource  availability  as  compared  with  a  monitoring  level  that  is 
associated  with  a  specific  monitoring  requirement  that  may  not  be 
achievable  with  existing  resources. 


O  T  TDP  Current  Activities  Since  April  2001,  OJJDP  has  been  engaged  in  a  review  of  its  monitoring 

practices.16  According  to  OJJDP  officials,  this  includes  a  thorough  review 
of  its  monitoring  practices,  including  improvement  of  its  grant  records. 
OJJDP  indicated  that  a  Working  Group  on  Monitoring  was  created  and 
charged  with  reviewing  its  revised  OJP  Handbook:  Policies  and 
Procedures  for  the  Administration  of  OJP  Grants  to  identify  new  and 
modified  policies  and  assess  the  need  for  training;  to  review  OJJDP’s  own 
policies  and  procedures  to  ensure  compliance  with  those  in  the  revised 
OJP  Handbook;  and  to  determine  whether  there  is  a  continued  need  for 
the  OJJDP  Monitoring  Desk  Book  (1991),  and  if  so,  how  it  should  be 
updated.  In  addition,  OJJDP  recently  embarked  on  an  overhaul  of  its  grant 
closeout  practices  to  improve  its  monitoring  records.  In  spring  2000,  it 
launched  a  grant  closeout  effort  jointly  with  the  Office  of  the  Comptroller 
that  is  intended  to  significantly  reduce  the  backlog  of  grants  requiring 
closeout.  As  part  of  this  activity,  grant  files  were  reviewed,  and  relevant 
documentation  was  collected. 


16As  of  August  2001,  OJJDP’s  efforts  were  ongoing. 
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Grant  Files  Contained 
Limited 

Documentation  That 
Monitoring  Occurred 


OJJDP  requires  its  grant  managers  to  document  their  monitoring  of 
discretionary  grants.  This  documentation  is  to  include  the  development 
and  retention  of  a  grant  manager’s  memorandum  for  each  grant  containing 
a  plan  to  follow  for  monitoring  the  project.  In  addition,  progress  reports 
documenting  progress  achieved  in  relation  to  project  milestones  are 
required  from  all  grantees.  Required  documentation  also  is  to  include 
quarterly  financial  reports  from  grantees.  In  addition,  OJJDP  requires 
certain  documentation  at  the  time  of  grant  closeouts. 


While  the  grant  files  generally  contained  monitoring  plans,17  OJJDP  grant 
managers  were  not  consistently  documenting  their  monitoring  efforts 
according  to  the  monitoring  plans.  The  grant  files  generally  did  not  contain 
documentation  showing  required  telephone  contacts  and  site  visits.18 
Further,  progress  and  financial  reports  did  not  cover  the  entire  grant 
period  over  half  of  the  time.  For  those  grants  in  our  sample  that  were 
closed,  most  did  not  contain  documentation  showing  compliance  with 
closeout  procedures.  We  identified  some  of  these  same  documentation- 
related  problems  in  May  1996. 

We  compared  the  proposed  monitoring  activities  in  the  grant  manager’s 
monitoring  plan  for  89  of  545  OJJDP  demonstration  discretionary  grant 
files  and  45  of  83  OJJDP  training  and  technical  assistance  discretionary 
grant  files  to  actual  monitoring  documentation.  Our  comparisons  of  the 
proposed  to  actual  documented  monitoring  revealed  that  OJJDP  grant 
managers  were  not  consistently  following  the  monitoring  plan. 

•  Telephone  Contacts.  The  grant  managers’  monitoring  plans  for  the  most 
recent  awards  for  79  demonstration  grants  and  34  training  and  technical 
assistance  grants  had  specific  requirements  for  telephone  contacts  (e.g., 
monthly  or  quarterly).  There  was  no  documentation  showing  that  the 
requirement  for  telephone  contacts  was  met  in  96  percent  of  the 
demonstration  grants  and  in  any  of  the  training  and  technical  assistance 
grants.  Documentation  was  present  that  showed  that  some,  but  not  all,  of 
the  required  telephone  contacts  had  been  made  for  49  percent19  of  the 


"While  required  for  every  award,  our  review  of  each  grant’s  most  recent  award  showed 
that  1  of  the  89  demonstration  grant  files  and  3  of  the  45  technical  assistance  grant  files 
were  missing  the  monitoring  plan. 

18The  absence  of  documentation  in  the  grant  file  does  not  necessarily  mean  that  monitoring 
did  not  occur. 

I9The  estimated  range  is  between  38  and  61  percent 
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demonstration  grants  and  41  percent20  of  the  training  and  technical 
assistance  grants. 

•  Site  Visits.  The  grant  managers’  monitoring  plans  for  the  most  recent 
awards  for  66  demonstration  grants  and  20  training  and  technical 
assistance  grants  had  a  specified  number  of  on-site  visits  to  be  made. 

There  was  no  documentation  showing  that  the  requirement  for  site  visits 
was  met  in  88  percent21  of  the  demonstration  grants  and  90  percent22  of  the 
training  and  technical  assistance  grants.  Documentation  was  present  that 
showed  that  some,  but  not  all,  of  the  required  on-site  visits  had  been  made 
for  6  percent  of  the  demonstration  grants  and  10  percent23  of  the  training 
and  technical  assistance  grants.  When  site  visits  are  made  to  grantees,  a 
written  site  visit  report  is  to  be  prepared.  Of  the  awards  that  had  any 
evidence  of  site  visits,  such  as  memos  or  e-mails  referring  to  the  visit,  7  out 
of  the  12  demonstration  grants  and  2  out  of  the  4  training  and  technical 
assistance  grants  had  any  of  the  OJP-required,  written  site  visit  reports. 

In  commenting  on  a  draft  of  this  report,  the  Acting  Assistant  Attorney 
General  informed  us  that  most  grant  managers  prepare  site  visit  reports 
after  an  on-site  review,  as  their  travel  vouchers  are  not  forwarded  for 
payment  unless  the  reports  are  attached.  She  noted  that  the  grant 
managers  do  not  routinely  forward  copies  of  the  reports  to  the  official 
grant  and  program  files.  She  said  that  a  copy  of  the  reports  is  filed  in  the 
grant  manager’s  travel  folders  that  we  did  not  review.  However,  according 
to  the  OJP  Handbook,  the  original  site  visit  report  is  to  be  placed  in  the 
official  grant  file.  While  the  grant  managers’  travel  folders  may  contain  site 
visit  reports,  the  official  grant  files  kept  by  the  OJP  Comptroller’s  Control 
Desk  are  the  official  record  of  the  history  and  activity  of  the  grant.  As 
such,  we  reviewed  the  official  grant  files  as  well  as  the  grant  managers’ 
files  for  required  grant  documents  and  not  the  travel  voucher  folders. 

In  another  comment  to  a  draft  of  this  report,  the  Acting  Assistant  Attorney 
General  told  us  that  monitoring  plans  that  call  for  on-site  visits  by  grant 
managers  are  contingent  upon  staffing  capacity  and  administrative 
resources  and  not  requirements.  Our  review  indicated  that,  contrary  to  the 


20The  estimated  range  is  between  25  and  59  percent. 
2iThe  estimated  range  is  between  77  and  95  percent. 
22The  estimated  range  is  between  68  and  99  percent 
23The  estimated  range  is  between  1  and  32  percent. 
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Acting  Assistant  Attorney  General’s  comment,  certain  specific  program 
monitoring  requirements  are  included  in  the  OJP  Handbook,  such  as 
creating  monitoring  plans  and  writing  site  visit  reports  within  10  days  of  a 
site  visit.  To  be  useful,  monitoring  plans  should  be  feasible.  If  they  are  not 
feasible,  then  they  will  not  be  helpful  guidance  for  monitoring.  While  it  is 
entirely  reasonable  to  expect  that  some  planned  site  visits  will  not  take 
place  (for  a  variety  of  reasons),  the  proportion  of  site  visits  specified  to 
occur  that  were  not  documented  seems  to  indicate  that  either  they 
occurred  but  were  not  documented  or  monitoring  plans  for  site  visits  were 
infeasible. 

•  Progress  and  Financial  Reports.  According  to  the  OJP  Handbook, 
recipients  of  discretionary  grants  are  required  to  submit  initial  and  then 
semiannual  categorical  assistance  progress  reports  documenting  progress 
achieved  in  relation  to  project  milestones.  Before  December  31, 1995, 
progress  reports  were  due  every  quarter.  Since  March  1, 1996,  progress 
reports  have  been  due  every  6  months.  This  report  must  be  submitted 
within  30  days  after  the  end  of  the  reporting  period  for  the  life  of  the 
award.  A  final  report,  which  provides  a  summary  of  progress  toward 
achieving  the  goals  and  objectives  of  the  award,  significant  results,  and 
any  products  developed  under  the  award,  is  due  90  days25  after  the  end 
date  of  the  award. 

We  found  that  in  56  percent  of  the  89  demonstration  grant  files  and  80 
percent26  of  the  45  training  and  technical  assistance  grant  files  reviewed, 
the  progress  reports  did  not  cover  the  entire  grant  period.  In  addition,  7 
percent  of  the  demonstration  grants  files  and  4  percent  of  the  training  and 
technical  assistance  grants  had  no  progress  reports. 

We  also  found  that  56  percent  of  the  progress  reports  submitted  for 
demonstration  grants  were  filed  on  time;  for  the  other  44  percent,  the 
number  of  days  late  ranged  from  1  to  545,  with  62  percent  of  the  reports 


24In  January  2001,  OJP  released  a  revised  Grant  Management  Policies  and  Procedures 
Manual  to  update  and  codify  OJP’s  current  policies  regarding  its  business  practices. 
According  to  OJP  officials,  this  document  superceded  and  canceled  OJP  HB  4500.2C, 
Policies  and  Procedures  for  the  Administration  of  OJP  Grants,  dated  February  19, 1992. 
The  revised  policies  and  procedures  were  not  applicable  during  the  period  of  our  review. 

25The  OJP  Grant  Management  Policies  and  Procedures  Manual  dated  January  2001, 
changed  the  final  progress  report  due  date  from  90  days  to  120  days.  The  revised  policies 
and  procedures  were  not  applicable  during  the  period  of  our  review. 

26The  estimated  range  is  between  69  and  88  percent 
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for  demonstration  grants  being  late  up  to  30  days  and  24  percent  between 
31  and  90  days.27  We  found  that  42  percent  of  the  training  and  technical 
assistance  grant  reports  were  filed  on  time;  for  the  other  58  percent,  the 
number  of  days  late  ranged  from  1  to  297,  with  76  percent  being  late  up  to 

30  days  and  15  percent  between  31  and  90  days. 

According  to  the  OJP  Handbook,  all  discretionary  grantees  are  required  to 
submit  quarterly  financial  status  reports  to  OJP’s  Office  of  the  Comptroller 
Control  Desk,  which  maintains  the  official  grant  files.  Copies  are 
disseminated  to  the  appropriate  financial  analysts  and  cognizant  grant 
managers.  These  quarterly  reports  are  required  for  all  active  grants,  even  if 
there  has  been  no  financial  activity  during  the  reporting  period.  According 
to  OJP  officials,  reports  are  considered  delinquent,  for  payment  purposes, 
if  not  received  45  days  after  the  end  of  each  calendar  quarter.  The  final 
financial  status  report  is  due  90  days28  after  the  end  of  the  grant  period  or 
extension.  Future  awards  and  fund  drawdowns  may  be  withheld  if  the 
progress  and  financial  status  reports  are  delinquent. 

We  found  that  in  65  percent  of  the  89  demonstration  grant  files  and  60 
percent29  of  the  45  training  and  technical  assistance  grant  files  reviewed, 
the  financial  status  reports  did  not  cover  the  entire  grant  period.  We  also 
found  that  67  percent  of  the  financial  status  reports  submitted  for 
demonstration  grants  were  filed  on  time;  for  the  other  33  percent,  the 
number  of  days  late  ranged  from  1  to  367,  with  54  percent  of  the  reports 
for  demonstration  grants  being  late  up  to  30  days  and  34  percent  between 

31  and  90  days.30  We  found  that  70  percent  of  the  training  and  technical 


27We  reviewed  all  321  demonstration  grant  progress  reports  and  252  training  and  technical 
assistance  grant  progress  reports  in  the  files  from  the  initiation  of  the  grant  through  the 
date  of  our  review  for  grants  that  were  active  in  all  of  fiscal  years  1999  and/or  2000.  We 
excluded  13  progress  reports  for  demonstration  grants  and  16  reports  for  training  and 
technical  assistance  grants  because  of  data-related  problems,  such  as  missing  submission 
dates. 

28The  OJP  Grant  Management  Policies  and  Procedures  Manual  dated  January  2001, 
changed  the  final  financial  status  report  due  date  from  90  days  to  120  days.  The  revised 
policies  and  procedures  were  not  applicable  during  the  period  of  our  review. 

29The  estimated  range  is  between  48  and  71  percent. 

30We  reviewed  all  615  demonstration  financial  status  reports  and  603  training  and  technical 
assistance  financial  status  reports  in  the  files  from  the  initiation  of  the  grant  through  the 
date  of  our  review  for  grants  that  were  active  in  all  of  fiscal  years  1999  and/or  2000.  We 
excluded  18  financial  status  reports  for  demonstration  grants  and  6  reports  for  training  and 
technical  assistance  grants  because  of  data-related  problems,  such  as  missing  submission 
dates. 
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assistance  grant  reports  were  filed  on  time;  for  the  other  30  percent,  the 
number  of  days  late  ranged  from  1  to  353,  with  72  percent  being  late  up  to 
30  days,  and  19  percent  between  31  and  90  days. 

Further,  our  review  found  no  documentation  in  the  files  that  OJJDP  grant 
managers  regularly  reviewed  these  reports,  as  required  by  OJP.  Without 
such  reviews,  OJJDP  does  not  have  assurances  that  the  grants  are  being 
properly  administered.  For  example,  one  grant  we  reviewed  had  no 
documentation  of  required  semiannual  categorical  assistance  progress 
reports  or  quarterly  financial  reports  in  the  file  for  the  entire  life  of  the 
award.  The  project  and  budget  periods  ran  from  October  1, 1999,  to 
September  30,  2000.  A  $300,000  grant  was  awarded  on  March  7,  2000  (6 
months  after  the  start  of  the  budget  period).  The  grantee  was  required  to 
submit  an  initial  and  then  semiannual  categorical  assistance  progress 
reports  documenting  progress  achieved  in  relation  to  project  milestones, 
starting  June  30,  2000,  and  every  6  months  thereafter.  The  grantee  was  also 
required  to  submit  a  quarterly  financial  status  report  within  45  days  after 
the  quarter  ended,  starting  May  15,  2000,  and  every  quarter  thereafter. 

Although  no  progress  or  financial  reports  had  been  submitted  during  the 
life  of  the  grant  period,  OJJDP  did  not  discover  this  until  almost  a  full  year 
from  the  award  date.  Nevertheless,  OJJDP  extended  the  grant  period  for 
an  additional  year  to  allow  the  grantee  to  spend  the  funds  for  the  project, 
which  it  had  not  drawn  down  during  the  grant  period.  Missing  progress 
reports  and  quarterly  financial  reports  increase  the  risk  that  OJJDP  might 
not  be  able  to  identify  and  rectify  grantee’s  program  and  financial 
problems  in  a  timely  manner. 

In  commenting  on  a  draft  of  this  report,  the  Acting  Assistant  Attorney 
General  acknowledged  this  lack  of  documentation,  but  did  not  agree  that 
it  meant  that  OJJDP  had  not  been  monitoring  grants  to  ensure  they  were 
being  properly  implemented.  From  our  perspective,  this  lack  of 
documentation  suggests  to  us  that  OJJDP  does  not  know  the  extent  to 
which  grant  managers  were  carrying  out  their  monitoring  responsibilities 
to  ensure  that  grants  were  being  properly  implemented. 

Despite  the  lack  of  evidence  that  grant  managers  were  adhering  to  the 
established  monitoring  plans,  we  did  find  that  some  grant  monitoring  was 
occurring.  For  example,  the  files  contained  evidence  that  communication 
occurred  between  grantees  and  the  grant  managers  in  the  form  of  reports, 
faxes,  letters,  and  oral  and  e-mail  communication  on  specific  issues, 
problems,  or  requests  for  information.  However,  much  of  the 
communication  between  grantees  and  grant  managers  was  not 
documented  according  to  OJJDP  requirements. 
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•  Closeout  Procedures.  The  OJP  grant  closeout  process  calls  for  an 
accounting  of  programmatic  accomplishments  in  relation  to  planned 
activities;  an  inventory  of  all  required  financial,  programmatic,  and  audit 
reports;  and  an  accounting  of  all  federal  funds.  According  to  the  OJP 
Handbook,  the  grant  manager  is  to  complete  an  assessment  report  and 
other  closure  requirements  within  180  days  after  the  end  date  of  the  award 
or  any  approved  extension.  These  requirements  include  a  closeout 
checklist,  overall  assessment  of  grant  performance,  final  progress  reports, 
final  financial  status  reports,  and  closeout  notification  and  grant 
adjustment  notice. 

Our  assessment  of  the  closeout  process  did  not  include  enough  cases  to 
ensure  that  it  is  representative  of  all  grant  closeouts.31  However,  our 
limited  review  showed  that  some  grant  files  did  not  contain  required 
closeout  materials. 

At  the  time  of  our  review,  the  following  documents  were  missing  from  the 
files: 

•  The  required  case  checklists  were  not  prepared  for  two  of  the  eight 
demonstration  grants  that  expired  and  required  closeout.  Similarly, 
neither  of  the  two  training  and  technical  assistance  grants  had  the 
required  case  checklists. 

•  An  overall  assessment  of  grant  performance  was  not  prepared  for  two 
of  the  eight  demonstration  grants.  Neither  of  the  two  training  and 
technical  assistance  grants  had  this  assessment. 

•  Final  progress  reports  were  missing  from  three  of  the  eight 
demonstration  grant  files.  Neither  of  the  two  training  and  technical 
assistance  grants  had  final  progress  reports  in  the  files. 

•  The  final  financial  status  report  was  missing  from  one  of  the  eight 
demonstration  grant  files.  Neither  of  the  two  training  and  technical 
assistance  grants  had  a  final  financial  status  report  in  the  files.  And 
there  was  no  documentation  that  grant  managers  had  reviewed  any  of 
the  reports  or  other  official  documents  that  were  filed. 


31Because  of  the  small  number  of  closeout  cases  in  our  sample  of  active  grants  in  all  of 
fiscal  years  1999  and/or  2000,  we  have  no  assurances  that  the  eight  demonstration  and  two 
training  assistance  grants  would  be  representative  of  all  closed  out  grants. 
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•  A  closeout  notification  and  grant  adjustment  notice  to  deobligate  funds 
or  advise  the  grantee  to  retain  records  for  3  years  was  not  prepared  for 
three  of  the  eight  expired  demonstration  grants.  Similarly,  neither  of 
the  two  training  and  technical  assistance  grants  had  a  closeout 
notification  and  grant  adjustment  notice. 

In  commenting  on  a  draft  of  this  report,  the  Acting  Assistant  Attorney 
General  noted  the  programmatic  and  fiscal  closure  process  might  not  have 
been  completed  for  those  grants  until  after  our  review.  In  our  review  of 
closeout  procedures,  we  waited  at  least  the  required  180  days  before 
reviewing  the  grant  files  to  allow  sufficient  time  for  OJJDP  to  complete  the 
grant  closeout  process.  However,  the  files  we  reviewed  did  not  contain  the 
required  closeout  documents. 

The  Comptroller  General’s  internal  control  standards32  require  that  all 
transactions  and  other  significant  events  be  clearly  documented  and  that 
the  documentation  be  readily  available  for  examination.  Appropriate 
documentation  is  an  internal  control  activity  to  help  ensure  that 
management’s  directives  are  carried  out.  Without  such  documentation, 
OJJDP  has  no  assurance  that  grants  are  meeting  their  goals  and  funds  are 
being  used  properly. 

In  commenting  on  a  draft  of  this  report,  the  Acting  Assistant  Attorney 
General  also  said  that  almost  50  percent  of  the  89  Special  Emphasis 
Division  grants  that  we  reviewed  were  awarded  in  fiscal  years  1998  and 
1999  under  the  Drug-Free  Communities  Support  Program.  Because  of  the 
time  needed  to  recruit,  hire,  and  train  new  program  staff,  OJJDP  handled 
217  Drug-Free  Communities  Support  Program  grants  in  those  2  years  with 
less  than  adequate  (i.e.,  two)  staff.  According  to  the  Acting  Assistant 
Attorney  General,  caseloads  of  this  size  make  proper  documentation  a 
difficult  goal  to  achieve. 


32 Standards  for  Internal  Control  in  the  Federal  Government  (GAO/AIMD-OO-2 1.3.1,  Nov. 
1999). 
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OJJDP  Does  Not 
Systematically 
Determine  Staff 
Compliance  With  Its 
Monitoring 
Requirements 


OJJDP  does  not  systematically  monitor  grant  managers’  compliance  with 
its  monitoring  requirements  or  guidance  nor  does  it  assess  the 
effectiveness  of  OJJDP  grant  monitoring  practices.  When  we  asked  OJJDP 
officials  about  any  oversight  of  OJJDP  grant  managers  to  determine  the 
adequacy  of  their  monitoring  of  OJJDP  discretionary  grants,  they  told  us 
oversight  takes  several  forms.  They  pointed  out  that  the  OJP  Handbook 
requires  staff  performance  to  be  carefully  scrutinized  against  the 
performance  work  plans  (i.e.,  the  critical  performance  elements  and 
standards  pertaining  to  monitoring  responsibilities).  They  said  that  OJJDP 
supervisory  managers  are  to  address  the  following  questions  to  determine 
how  well  each  grant  manager  with  monitoring  responsibilities  is 
performing:  (1)  how  informed  is  the  grant  manager  about  his  or  her 
project;  (2)  what  specific  oversight  and  review  activities  are  being 
conducted;  and  (3)  how  are  grantee  performance  issues,  if  any,  being 
resolved? 


Our  review  of  the  performance  work  plan  OJJDP  considered  to  be 
representative  found  that  it  tended  to  contain  information  that  made  it 
difficult  to  precisely  understand  the  nature  of  the  monitoring  to  be 
conducted.  The  performance  work  plan,  for  example,  referred  to 
monitoring  grants  to  ensure  that  projects  meet  their  goals  and  milestones, 
with  no  clear  expectations  regarding  monitoring  requirements,  such  as  site 
visits  or  telephone  monitoring,  and  documentation  of  these  contacts  and 
reviews  of  grantees’  progress  and  financial  reports. 

In  addition,  according  to  OJJDP  officials,  they  do  not  have  an  officewide 
management  information  system  that  tracks  monitoring  activities  specific 
to  each  grant  and  links  these  to  grant  monitoring  staff.33  However,  OJJDP 
officials  said  they  take  several  actions  to  ensure  that  staff  are  adequately 
performing  their  monitoring  responsibilities.  They  said  the  grant 
manager’s  memorandum  is  carefully  scrutinized  to  determine  that  the 
proper  information  regarding  grant  goals,  objectives,  tasks,  program 
activities,  and  products  are  included.  Division  directors  are  to  review 
requests  for  site  visits  on  a  quarterly  or  semiannual  basis  as  part  of  their 
development  of  travel  budgets.  Site  visit  reports  are  reviewed  at  the  time 
or  within  a  reasonable  period  of  submission  of  travel  reimbursement 
requests.  Finally,  division  directors  are  routinely  consulted  regarding 
corrective  action  on  grants. 


33OJP’s  Office  of  the  Comptroller  has  developed  an  automated  tracking  system  that  flags 
missing  and  late  progress  and  financial  reports.  This  information  is  available  to  OJJDP 
grant  managers.  However,  OJJDP  does  not  have  a  system  for  tracking  telephone  contacts, 
site  visits,  review  of  progress  and  financial  reports,  or  other  reporting  requirements. 
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Conclusions 


Recommendations  for 
Executive  Action 


In  our  May  1996  work,  we  reported  that  OJJDP’s  files  for  discretionary 
grants  generally  contained  monitoring  plans  but  little  documentation  that 
monitoring  occurred.  OJJDP  said  that  a  heavy  workload  might  have 
resulted  in  a  lack  of  monitoring  records,  but  that  it  would  take  the  steps 
necessary  to  improve  records.  OJJDP  has  made  limited  progress  in 
addressing  the  monitoring  problems  we  identified  in  1996. 

Our  review  showed  that  monitoring  activities  continue  to  be  insufficiently 
documented,  and  because  of  this,  neither  OJJDP  nor  we  can  determine  the 
level  of  monitoring  being  performed  by  grant  managers  as  required  by 
OJP,  OJJDP,  and  the  Comptroller  General’s  internal  control  standards. 

OJJDP  also  cannot  rely  on  supervisory  oversight  of  its  grant  managers  to 
ensure  that  monitoring  is  being  performed  as  required.  The  use  of  staff 
performance  work  plans  seems  problematic  as  a  means  of  reviewing 
monitoring  activities  because  the  requirements  are  written  in  such  general 
terms.  Grant  managers’  performance  work  plans  do  not  include  clear 
expectations  and  accountability  for  monitoring  called  for  in  grant 
managers’  monitoring  plans  as  well  as  follow-up  requirements  to  notify 
and  assist  grantees  who  consistently  submit  late  or  no  progress  or 
financial  reports.  Further,  the  approval  of  grant  managers’  memorandums 
or  preapproval  of  site  visits  does  not  ensure  that  required  monitoring  is 
taking  place,  and  none  of  these  oversight  activities  is  designed  to  ensure 
that  proper  documentation  of  monitoring  activity  is  taking  place. 

Current  activities  to  review  monitoring  policies  and  practices  and  to 
complete  documentation  for  closed  grants  could  result  in  positive 
changes.  However,  these  activities  are  too  new  to  be  evaluated  in  relation 
to  their  potential  for  improving  grant  monitoring  or  its  documentation. 


The  current  review  of  monitoring  policies  and  practices  affords  an 
opportunity  for  OJJDP  to  assess  monitoring  in  light  of  our  recent  findings. 
To  facilitate  and  improve  the  management  of  program  grant  monitoring, 
we  recommend  that  the  Attorney  General  direct  OJJDP,  as  part  of  its 
ongoing  review  of  monitoring  policies  and  practices,  to  determine  whether 
the  monitoring  documentation  problems  that  we  identified  were  an 
indication  of  grant  monitoring  requirements  not  being  met  or  of  a  failure 
to  document  activities  in  the  official  grant  files  that  did,  in  fact,  take  place. 

If  monitoring  requirements  are  not  being  met,  we  recommend  that  the 
Attorney  General  direct  OJJDP  to  determine  why  this  is  so  and  to  consider 
those  reasons  as  it  develops  solutions  for  improving  compliance  with  the 
requirements. 
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If  it  is  determined  that  required  monitoring  is  taking  place  but  is  not  being 
documented,  we  recommend  that  the  Attorney  General  direct  OJJDP  to 
develop  and  enforce  clear  expectations  regarding  monitoring 
requirements.  This  policy  should,  among  other  things,  require  supervisory 
review  to  ensure  that  monitoring  activities  are  being  carried  out  and 
documented  as  prescribed.  Further,  grant  managers’  performance  work 
plans  should  include  clear  expectations  and  accountability  for  monitoring 
called  for  in  grant  managers’  monitoring  plans  as  well  as  follow-up 
requirements  to  notify  and  assist  grantees  who  consistently  submit  late  or 
no  progress  or  financial  reports. 


Agency  Comments 
and  Our  Evaluation 


We  provided  a  copy  of  this  report  to  the  Attorney  General  for  review  and 
comment.  In  a  September  19,  2001,  letter,  the  Acting  Assistant  Attorney 
General  commented  on  a  draft  of  this  report.  Her  comments  are 
summarized  below  and  are  presented  in  their  entirety  in  appendix  n. 

The  Acting  Assistant  Attorney  General  said  that  overall,  the  report 
provides  useful  information  in  highlighting  management  and  monitoring 
activities  in  need  of  improvement.  Senior  staff  within  OJJDP  have  been 
directed  to  address  each  of  the  issues  identified  in  the  draft  report  to  focus 
on  adjusting  the  nature  and  types  of  monitoring  activities  pursuant  to 
these  types  of  grants,  thus  ensuring  that  every  active  grant  receives  a 
programmatic  assessment.  OJJDP  will  also  be  considering  ways  to 
improve  and  streamline  grant  management  monitoring  activities  to  ensure 
that  program  monitoring  efforts  are  documented  and  useful,  including  the 
development  of  a  management  information  system  and  the  use  of  private 
contractors  to  assist  in  some  of  these  review  functions. 

The  Acting  Assistant  Attorney  General  also  said  that  the  draft  report  could 
have  gone  further  in  acknowledging  the  impact  of  OJJDP’s  enormous 
growth  of  the  past  5  years  on  its  ability  to  monitor  recipients.  We 
recognized  that  the  number  of  active  OJJDP  discretionary  grants  has  more 
than  tripled  over  the  past  5  years.  However,  it  is  OJJDP’s  responsibility  to 
initiate  action,  such  as  to  request  additional  funding  to  meet  its  monitoring 
responsibilities  or  revise  its  monitoring  efforts  to  comport  with  its 
available  resources.  In  commenting  on  our  May  1996  testimony,  the  OJJDP 
Deputy  Administrator  recognized  that  OJJDP’s  heavy  workload  might  have 
resulted  in  a  lack  of  monitoring  records,  but  that  he  would  take  steps 
necessary  to  improve  records.  However,  our  current  review  disclosed  that 
similar  monitoring  problems  persist. 

Further,  the  Acting  Assistant  Attorney  General  said  that  we  did  not 
acknowledge  OJJDP’s  efforts  in  a  major  OJP-wide  effort  to  improve  grant 
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monitoring  practices  and  establish  uniform  policies.  We  recognized  some 
of  the  efforts  to  review  its  monitoring  practices,  such  as  revising  the  OJP 
Handbook:  Policies  and  Procedures  for  the  Administration  of  OJP 
Grants  and  its  recent  embarking  on  an  overhaul  of  its  grant  closeout 
practices  to  improve  its  monitoring  records.  Such  efforts,  we  believe, 
should  help  address  the  monitoring  problems  we  identified. 

The  Acting  Assistant  Attorney  General  concurred  with  our 
recommendation  that  grant  managers’  performance  work  plans  include 
clear  expectations  and  accountability  for  monitoring.  She  said 
performance  work  plans  put  into  effect  after  February  2001  for  grant 
managers  in  OJJDP’s  Special  Emphasis  Division  more  clearly  reflect 
monitoring  responsibilities  and  provide  for  better  supervisory  controls. 
Further,  she  noted  that  OJJDP’s  Special  Emphasis  Division  has  put  into 
place  a  new  process  designating  one  staff  member  with  responsibility  for 
file  “audits”  to  help  ensure  that  files  are  in  order  and  that  documentation 
of  monitoring  is  in  place. 


As  agreed  with  your  office,  unless  you  publicly  announce  the  contents  of 
this  report  earlier,  we  plan  no  further  distribution  of  it  until  30  days  from 
the  date  of  this  letter.  At  that  time,  we  will  send  copies  to  the  Chairmen 
and  Ranking  Members  of  the  Senate  Judiciary  Committee;  Senate 
Subcommittee  on  Youth  Violence;  House  Committee  on  Education  and  the 
Workforce;  House  Subcommittee  on  Early  Childhood,  Youth  and  Families; 
Attorney  General;  and  Director,  Office  of  Management  and  Budget.  We  will 
also  make  copies  available  to  others  upon  request. 

If  you  or  your  staff  have  any  questions  about  this  report,  please  contact  me 
or  James  M.  Blume  at  (202)  512-8777.  Key  contributors  to  this  report  are 
acknowledged  in  appendix  III. 

Sincerely  yours, 


Laurie  E.  Ekstrand 
Director,  Justice  Issues 
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Appendix  I:  OJJDP’s  Discretionary  Grant 
Award  Process 


Each  year,  OJJDP  publishes  a  proposed  program  plan  seeking  public 
comment  about  proposed  discretionary  funding  opportunities  for  activities 
covered  under  parts  C  and  D  (all  national  programs  and  gang-related 
activities)  of  the  Juvenile  Justice  and  Delinquency  Prevention  Act.  The 
proposed  plan  is  published  in  the  Federal  Register.  Program 
announcements  and  solicitations  for  discretionary  funding  opportunities 
are  published  as  they  become  available.  The  announcements  and 
solicitations  provide  details  about  specific  funding  opportunities, 
including  eligibility  requirements  and  deadlines.  These  include  funding 
opportunities  for  tribal  youth  and  drug-free  communities  as  well  as 
funding  opportunities  associated  with  parts  C  and  D.  Once  discretionary 
grant  applications  are  submitted,  OJJDP  said  that  its  contractor1  conducts 
an  initial  review  to  eliminate  any  applications  that  do  not  comply  with  the 
specified  grant  instructions.  After  ineligible  grant  applications  are 
eliminated,  OJJDP  has  a  peer  review  panel,  which  consists  of  three  or 
more  outside  experts  who  evaluate  and  rank  OJJDP’s  grant  applications. 
After  the  peer  review  panel  has  ranked  the  applications,  OJJDP’s  grant 
managers2  do  an  in-depth  review  of  applications  and  make  formal 
recommendations  regarding  the  funding  of  individual  grants  to  the  OJJDP 
Administrator. 

As  shown  in  figure  1,  the  key  parts  of  OJJDP’s  process  for  planning  and 
announcing  discretionary  grants  are  issuance  of  a  program  plan,  issuance 
of  discretionary  program  announcements,  and  distribution  of  application 
kits. 


^JJDP’s  current  contractor  is  the  Juvenile  Justice  Research  Center  at  Aspen  Systems. 

2OJJDP’s  grant  managers  are  OJJDP  officials  with  responsibility  for  overseeing  and 
monitoring  grants. 
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Appendix  Is  OJJDP’s  Discretionary  Grant 
Award  Process 


Figure  1:  Key  Steps  in  OJJDP’s  Process  for  Planning  and  Announcing  Grants 


Source:  GAO  analysis  of  OJJDP  documents  and  interviews  with  OJJDP  staff. 


As  shown  in  figure  2,  according  to  OJJDP  officials,  the  key  parts  of  the 
grant  application  and  review  process  include  initial  screening  of  an 
application  by  the  OJJDP  contractor,  peer  review  of  applications,  and  the 
final  selection  of  applications  for  funding  by  the  OJJDP  Administrator  or 
Assistant  Attorney  General. 
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Appendix  I:  OJJDP’s  Discretionary  Grant 
Award  Process 


Figure  2:  Overview  of  OJJDP  Grant  Application  and  Review  Process 


aSome  program  managers  will  do  initial  screening  only  for  specific  programs. 
"Quality  control  checks  are  done  by  program  managers  before  the  peer  review. 
Source:  GAO  analysis  of  OJJDP  documents  and  interviews  with  OJJDP  staff. 
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Appendix  I:  OJJDP’s  Discretionary  Grant 

Award  Process 

Program  Plan  Is  to 
Describe  Goals  and 
Priorities 

OJJDP’s  proposed  program  plan  describes  OJJDP’s  goals  and  priorities 
and  the  particular  program  activities  it  plans  to  carry  out  during  the  fiscal 
year  under  parts  C  and  D  of  Title  II  of  the  OJJDP  Act.  OJJDP  publicly 
announces  its  proposed  program  plan  in  the  Federal  Register.  After  a  45- 
day  comment  period,  OJJDP  is  to  publish  a  final  plan3  that  is  to  take  into 
consideration  comments  received  during  the  comment  period. 

According  to  an  official,  the  OJJDP  program  plan  is  generally  drafted  by 
the  Administrator  and  senior  managers  and  based  on  (1)  congressional 
and  departmental  priorities,  (2)  their  knowledge  and  expertise,  and  (3) 
input  during  the  course  of  the  year  from  a  variety  of  sources.  These 
sources  include  program  reviews  by  OJJDP  staff  and  comments  from 
practitioners  in  the  field,  officials  from  Justice  and  OJP  bureaus  and 
offices,  and  other  federal  agencies. 

Program 

Announcements  and 
Application  Kits  Are 
to  Solicit  Applicants 
for  Programs 

Following  publication  of  the  final  program  plan,  OJJDP  is  to  publish  in  the 
Federal  Register  a  notice  of  discretionary  grant  programs  and  to  announce 
the  availability  of  an  application  kit  for  grants.  The  notice  is  to  include  the 
availability  of  funds,  criteria  for  selection  of  grant  applicants,  procedures 
applicable  to  the  submission  and  review  of  application  for  submission  and 
review  of  applications  for  assistance,  and  information  on  how  to  obtain  an 
application  kit. 

Peer  Review  Panels 
Are  to  Help 

Determine  Which 
Grant  Applications 
Are  Funded 

Upon  receipt  of  discretionary  grant  applications,  the  OJJDP  contractor 
does  an  initial  review  of  each  application,  using  an  application  review 
checklist.  The  objective  of  this  initial  review  is  to  determine  whether 
applications  are  complete  and  eligible  for  federal  funding. 

OJJDP  contracts  out  the  administrative  arrangements  (e.g.,  arranging 
panels  and  notifying  reviewers)  for  the  peer  review  of  its  discretionary 
grant  applications.  The  contractor  maintains  a  list  of  qualified  consultants 
from  which  peer  reviewers  were  selected  by  OJJDP.  An  OJJDP  official 
estimated  that  there  were  73  peer  review  panels  in  fiscal  year  2000  for  its 

22  grant  programs.  The  grant  managers  identify  the  particular  expertise 
they  want  the  peer  reviewer  to  have.  The  contractor  generates  a  list  of 
potential  reviewers  with  their  resumes,  background  information,  and  other 
helpful  information.  The  grant  managers  recommend  peer  reviewers  on 

3The  plan  is  to  be  published  before  December  31  each  fiscal  year. 
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Award  Process 


the  basis  of  the  contractor’s  list  and  give  the  list  to  the  Deputy 
Administrator  for  approval.4  After  the  Deputy  Administrator  approves  the 
list,  it  is  sent  to  the  Administrator.  Once  the  Administrator  selects  the  peer 
reviewers,  the  contractor  makes  arrangements  with  the  individuals 
selected  and  manages  the  peer  review  process. 

According  to  an  OJJDP  official,  consultants  performing  the  peer  reviews 
are  reimbursed  at  a  flat  rate  of  $150  a  day.  OJJDP  guidance  requires  that 
the  grant  managers  use  the  following  criteria  to  help  achieve  balance  on 
the  peer  review  panel. 

•  Each  reviewer  should  have  expertise  in  or  complementary  to  the  subject 
area  under  review. 

•  Where  possible,  the  peer  review  panel  should  be  composed  of  a  mix  of 
researchers,  practitioners,  and  academicians. 

•  Special  attention  should  be  paid  to  obtaining  qualified  representation  from 
women  and  minority  groups. 

OJJDP’s  peer  review  process  is  advisory  and  is  to  supplement  and  assist 
OJJDP’s  consideration  of  applications.  However,  an  OJJDP  official  noted 
that,  in  practice,  results  of  the  peer  review  panels’  consideration  of 
application  were  almost  always  accepted. 


Final  Decision  to 
Award  Discretionary 
Grants 


After  completion  of  the  peer  review  panels,  OJJDP’s  grant  managers  do  an 
in-depth  review  of  the  peer  review  panels’  ranked  list.  The  OJJDP  grant 
managers  make  recommendations  concerning  applications  to  receive 
funding  in  a  memorandum  to  the  Administrator,  and  the  Administrator  is 
to  make  tentative  grant  selections.  The  OJJDP  Administrator  makes  final 
awards  for  research  and  evaluation  grants,  while  final  awards  for  all 
training  and  technical  assistance  and  demonstration  grants  are  made  by 
the  Assistant  Attorney  General.  After  the  applications  are  approved,  OJP’s 
Office  of  the  Comptroller  does  a  financial  review  of  these  applications  to 
determine  whether  the  applicant  has  the  necessary  resources  and  integrity 


4The  Juvenile  Justice  and  Delinquency  Prevention  Act  of  1974,  as  amended,  requires  OJJDP 
to  have  formal  peer  review  process  for  discretionary  grant  awards  (National  Programs).  42 
U.S.C.  5601,  5665a  (d).  However,  OJJDP  chooses  to  require  all  the  competitive 
discretionary  grant  applications  to  be  involved  in  its  peer  review.  Noncompetitive  grants, 
earmarks,  and  uniquely  qualified  grants  do  not  go  through  a  formal  peer  review  process. 
OJJDP  usually  has  a  maximum  of  three  grants  that  are  uniquely  qualified,  which  are 
reviewed  and  approved  by  the  Administrator. 
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to  account  for  and  administer  federal  funds  properly  and  whether  budget 
and  cost  data  in  the  application  were  allowable,  effective,  and  reasonable. 
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Note:  GAO  comments 
supplementing  those  in 
the  report  text  appear  at 
the  end  of  this  appendix. 


U.S.  Department  of  Justice 

Office  of  Justice  Programs 


Office  of  the  Assistant  Attorney  General  Washington,  D.C.  20531 

SEP  1  9  2001 

Laurie  E.  Ekstrand 
Director 
Justice  Issues 

Tax  Administration  and  Justice 
General  Accounting  Office 
441  G  Street,  N.W.  Room  2A38 
Washington,  D.C.  20548 

Dear  Ms.  Ekstrand: 

This  correspondence  responds  to  the  General  Accounting  Office  (GAO)  draft  report 
entitled  “JUVENILE  JUSTICE:  Better  Documentation  of  Discretionary  Grant 
Monitoring  Needed  "  (GAO-01 -1044).  The  draft  report  has  been  reviewed  by  the  Office 
of  Justice  Programs*  (OJP’s)  Office  of  Juvenile  Justice  and  Delinquency  Prevention 
(OJJDP)  and  Office  of  the  Comptroller  (OC). 

Overall,  the  report  provides  useful  information  in  highlighting  management  and 
monitoring  activities  in  need  of  improvement.  Senior  staff  within  OJJDP  have  been 
directed  to  address  each  of  the  issues  identified  in  the  draft  report  to  focus  on  adjusting 
the  nature  and  types  of  monitoring  activities  pursuant  to  these  types  of  grants,  thus 
ensuring  that  every  active  grant  receives  a  programmatic  assessment.  The  OJJDP  will 
also  be  considering  ways  to  improve  and  streamline  grant  management  monitoring 
activities  to  ensure  that  program  monitoring  efforts  are  documented  and  useful,  including 
the  development  of  a  management  information  system  and  the  use  of  private  contractors 
to  assist  in  some  of  these  review  functions. 

However,  the  draft  report  could  have  gone  further  in  acknowledging  the  impact  of 
OJJDP ’s  enormous  growth  over  the  past  five  years  on  its  ability  to  monitor  recipients  and 
suggesting  ways  in  which  OJJDP  can  better  monitor  programs  with  current  resources.  In 
addition,  the  draft  report  does  not  acknowledge  OJJDP’s  efforts  in  a  major  OJP-wide 
effort  to  improve  grant  monitoring  practices  and  establish  uniform  policies.  Our  specific 
comments  to  the  draft  report  are  stated  below. 
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See  comment  1 . 


See  footnote  12. 


Specific  Comments 

1 .  Page  3,  Last  paragraph :  u Progress  and financial  reports  filed  by  the  grantee  are 
required  for  the  entire  grant  period.  ” 

While  grantees  are  required  to  submit  quarterly  Financial  Status  Reports 
(SF269(a))j  there  is  no  Federal  or  OJP  grant  requirement  that  recipients  have 
reports  covering  every  quarterly  reporting  period  in  their  grant  file.  The  practical 
reality  of  federal  grant  administration  is  that  grantees  will  occasionally  file  their 
reports  late  or  fail  to  file  a  quarterly  report.  Our  financial  controls  are  very 
stringent  in  this  regard,  as  no  payments  or  new  awards  can  be  made  if  the  recipient 
is  delinquent  in  its  financial  reporting,  thus  fulfilling  the  objective  GAO’s  internal 
control  standards. 

When  such  lapses  occur,  OJP  allows  its  recipients  to  file  a  “catch-up”  report 
bringing  their  cumulative  expenditure  and/or  obligation  information  up  to  date. 
OJP’s  financial  auditors  review  recipient  financial  reporting  as  apart  of  OJP’s 
annual  financial  audit  and  have  found  no  significant  weaknesses  in  this  regard. 
These  filings  are  subjected  to  especially  intense  scrutiny  during  the  review  of 
OJP’s  annual  accrual  of  grantee  expenditures. 

2.  Page  6, 1st  paragraph:  “ Four  grant files  were  never  found. " 

The  report,  as  written,  may  mislead  the  reader  into  believing  that  grant  files  were 
not  maintained  as  required.  We  suggest  that  the  draft  report  be  revised  to 
accurately  reflect  the  difficulty  encountered  in  locating  the  four  grant  files  that 
GAO  contends  “. . .  were  never  found  ” 

There  were  legitimate  reasons  for  the  official  grant  files  not  being  accessible 
during  the  review.  Specifically,  the  official  grant  file  for:  a)  grant  number  1999- 
JW-VX-K002  was  not  provided  due  to  an  incorrect  grant  number  (1999-JW-VX- 
0001)  on  GAO’s  list  provided  to  the  OC’s  Control  Desk  when  die  official  grant 
files  were  requested;  b)  grant  number  98-JN-FX-0019  was  not  provided  because 
the  grant  was  closed  in  March  2000  and  thus,  the  official  grant  file  had  been  sent 
to  off-site  storage;  and  c)  grant  number  1999-JS-FX-0024  was  not  provided 
because  the  official  grant  file  was  checked  out  by  OJP  staff  at  the  time  of  GAO’s 
review  -  unfortunately  the  user  failed  to  note  the  check-out  as  is  required,  which 
would  have  facilitated  its  location.  The  file  is  now  appropriately  filed  at  the  OC 
Control  Desk.  The  remaining  official  grant  file  was  inexplicably  missing,  but  is 
currently  being  reconstructed.  OJP  manages  nearly  45,000  active  awards.  The 
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Now  on  p.  8. 


Now  last  paragraph. 


fact  that  one  file  was  checked  out  and  one  file  was  missing  is  not  an  indication  of 
a  major  control  weakness,  especially  when  the  agency  has  redundancy  controls 
that  enable  file  recovery,  as  is  being  done  in  this  case.  The  audit  report,  as 
written,  may  unnecessarily  mislead  and  alarm  the  readers  of  this  report. 
Experienced  auditors  know  that  such  conditions  are  a  normal  by-product  of  any 
sampling  routine,  and  merely  must  be  taken  into  consideration  when  determining 
the  extent  to  which  sample  results  can  be  extrapolated  to  the  universe. 

GAO  has  amply  demonstrated  and  OJJDP  has  fully  acknowledged  its  failure  to 
adequately  document  its  program  monitoring  planning  and  execution.  The 
manner  in  which  this  inconsequential  sampling  administration  issue  is  currently 
portrayed  in  the  draft  report  detracts  from  the  professional  presentation  of 
legitimate  issues  that  all  agree  are  in  need  of  attention. 

3 .  Page  7,  3rd paragraph:  " The  1991  OJJDP  Monitoring  Desk  Book  contains 
specific  monitoring  procedures  that  grant  managers  are  to  use  for  monitoring 
OJJDP ’s  discretionary  grants.  ” 

This  Desk  Book,  which  was  developed  by  the  OJJDP’ s  Training  and  Technical 
Assistance  Division,  was  never  adopted  as  an  OJJDP  discretionary  grants 
monitoring  guide,  thus  is  not  a  valid  criterion  for  the  GAO  review.  The  grant 
management  guidance  used  for  monitoring  OJJDP  discretionary  grants  are 
outlined  in  the  OJP  Handbook:  Policies  and  Procedures  for  the  Administration  of 
OJP  Grants ,  February  1992,  and  the  OC’s  Financial  Guide.  The  OJP  Handbook 
was  updated  in  February  2001. 

4.  Page  8,  2nd  paragraph,  1st  sentence:  “To  assess  a  grantee ’s  financial  records ,  the 
Comptroller 's  Monitoring  Division  performs  an  in-house  review  of  various 
programs  and  financial  documents  contained  in  the  official  grant  file.  ” 

This  sentence  does  not  accurately  portray  the  objective  of  the  OC-based  review. 
Such  reviews  do  not  directly  assess  the  grantee’s  financial  records.  Rather,  such 
reviews:  Provide  as  comprehensive  a  financial  review  as  is  possible  without 
physically  going  on  site.  Such  reviews  are  one  way,  among  others,  that  the 
Comptroller’s  Monitoring  Division  ensures  grantee  compliance  with  financial 
reporting,  cash  management,  and  audit  reporting  requirements.  They  are  more  in 
depth  than  simple  checklists  or  desk  reviews.  MD  staff  perform  these  in-house 
reviews  by  examining  and  analyzing  various  program  and  financial  documents 
contained  in  official  grant  files  and  information  in  our  accounting  system,  and 
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Now  on  p.  9. 


Now  on  p.  9,  footnote  14. 


interviewing  cognizant  program  and  grantee  staff  regarding  project  progress  and 
any  issues  needing  attention.” 

As  this  is  only  one  part  of  OJP’s  comprehensive  financial  monitoring  program, 
GAO’s  footnote  treatment  of  OJP’s  on-site  financial  monitoring  should  be 
elevated  to  the  narrative  to  provide  a  more  balanced  presentation  of  die  agency’s 
proactive  financial  monitoring  activities. 

5.  Page  8,  2nd paragraph,  last  sentence:  “These  included  missing  program  and 
financial  documents  and  program  and  financial  monitoring  not  always  being 
documented  in  a  timely  manner. 

There  was  no  mention  of  delinquent  documentation  of  financial  monitoring 
during  GAO’s  review,  nor  was  there  any  evidence  cited  in  the  draft  report  to 
support  the  contention  that  financial  monitoring  was  not  documented  timely.  In 
fact,  OJP  has  an  effective  and  efficient  financial  monitoring  program  administered 
by  OC  as  noted  by  GAO  in  the  second  paragraph  on  page  8  and  footnote  13  of  the 
draft  report.  This  process,  as  well  as  OC’s  follow  up  on  related  corrective  actions 
has  been  subjected  to  audit  annually  by  our  financial  statement  auditors  who  have 
been  more  than  satisfied  with  our  comprehensive  financial  monitoring  program. 
OC -based  reviews  and  site  visit  reports  are  promptly  completed  and  distributed  to 
cognizant  program  office  staff  and  our  financial  auditors  have  specifically  tested 
our  report  production  and  related  follow  up. 

6.  Page  8,  Footnote  15:  “In  October  2000 ,  the  Comptroller  eliminated  its  review  of 
progress  reports  because  Comptroller  officials  believed  that  it  was  taking  a  lot  of 
time  to  try  to  track  these  reports  down  and  a  lot  of  their  desk  reviews  were  open 
because  the  Comptroller  could  not  locate  these  progress  reports 

On  September  12, 2001,  GAO  agreed  to  revise  the  statement  to  reflect  the  written 
response  provided  by  the  OJP  Audit  Liaison  on  August  1, 2001.  The  written 
response  stated  that  follow  up  on  progress  reports  was  eliminated  from  the  OC- 
based  review  guide  to  streamline  the  process  and  reduce  duplicative  monitoring 
between  the  program  offices  and  OC. 
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Now  on  p.  1 1 . 


See  p.  17. 


See  pp.  12-13. 


Now  on  p.  14. 


7.  Page  10,  2nd  paragraph:  “The  grant  files  generally  did  not  contain  documentation 
showing  required  telephone  contacts  and  site  visits.18” 

Page  10,  Footnote  18:  “ The  absence  (or  presence)  of  documentation  in  the  grant 
file  does  not  necessarily  mean  that  monitoring  did  not  (or  did)  occur.  ” 

We  agree  that  grant  files  generally  did  not  contain  documentation  of  program 
monitoring.  However,  the  lack  of  documentation  does  not  mean  that  monitoring 
did  not  occur.  Also,  we  believe  that  the  footnoted  statement  should  be  made  a 
part  of  the  text,  with  one  exception:  the  reference  in  the  footnote  that 
documentation  itself  does  not  mean  that  monitoring  did  occur  should  be  deleted  as 
this  seems  to  unfairly  imply  that  perhaps,  OJJDP  would  falsify  documentation  to 
indicate  evidence  of  monitoring  activities  when  none  occurred. 

In  addition,  it  appears  that  almost  50%  of  the  89  Special  Emphasis  Division  grants 
reviewed  by  GAO  were  awarded  in  Fiscal  Years  1998  and  1999  under  the  Drug 
Free  Community  (DFC)  Support  Program.  Because  of  the  time  needed  to  recruit, 
hire,  and  train  new  program  staff,  OJJDP  handled  the  (217)  1998  and  1999  DFC 
grants  with  less  than  adequate  (two)  professional  staff.  Caseloads  of  this  size 
make  proper  documentation  a  difficult  goal  to  achieve. 

8.  Page  11,2nd  paragraph:  “The  grant  managers  ’  monitoring  plans  for  the  most 
recent  awards  for  the  66  demonstration  grants  and  20  training  and  technical 
assistance  grants  had  specific  requirements  for  on-site  visits. ” 

The  draft  report  misrepresents  that  monitoring  plans  proposed  in  the  Grant 
Manager’s  Memoranda  were  “requirements”  and  not  “plans  contingent  upon 
staffing  capacity  and  administrative  resources. 

9.  Page  12,  Last  paragraph:  Reports  are  considered  delinquent  if  not  received 
within  45  days  after  the  end  of  each  calendar  quarter.  ” 

This  statement  should  be  revised  to  state  that  “Reports  are  considered  delinquent 
[for  payment  purposes]  if  not  received  within  45  days  after  the  end  of  each 
calendar  quarter.” 
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See  p.  14,  footnote  28. 


Now  on  p.  15. 


Now  on  p.  16,  4th  bullet. 


Now  on  p.  16,  1st  bullet. 


10.  Page  12,  Last  paragraph:  " The  final  financial  status  report  is  due  within  90  days 
after  the  end  of  the  grant  period,  or  extension.  ” 

This  statement  should  be  revised  to  accurately  state  that  “The  final  financial  status 
report  is  due  within  f  1201  days  after  the  end  of  the  grant  period,  or  extension.” 

11.  Page  15,2nd.  paragraph:  " Further,  our  review  found  no  documentation  in  the  files 
that  OJJDP  grant  managers  regularly  reviewed  these  [financial  and  progress] 
reports,  as  required  by  OJP.  Without  such  reviews,  OJJDP  does  not  have 
assurances  that  the  grants  are  being  properly  administered  " 

While  we  acknowledge  this  lack  of  documentation,  we  do  not  agree  that  it  means 
that  OJJDP  has  not  been  monitoring  grants  to  ensure  they  are  being  properly 
implemented. 

12.  Page  14,  1st  bullet:  “At  the  time  of  our  review,  two  of  the  eight  demonstration 
grantees  had  not filed final financial  status  reports.  Neither  of  the  two  training 
and  technical  assistance  grants  had  this  final financial  status  report” 

On  September  12,  2001,  GAO  stated  that  the  second  sentence  should  reflect  that 
one  (instead  of  two)  of  the  training  and  technical  assistance  grants  did  not  have 
this  final  financial  status  report. 

Based  on  our  review  of  the  closeout  files  for  the  three  grants  GAO  is  referring  to 
in  the  draft  report,  closeout  action  was  not  completed  for  these  grants  until  after 
GAO’s  March  2001  file  review.  Thus,  although  they  were  not  in  the  file  when 
GAO  conducted  their  file  review,  as  part  of  the  programmatic  and  fiscal  closure 
process,  OJP  obtains  a  final  Financial  Status  Report  and  required  programmatic 
reports,  and  those  reports  and  other  closeout  documentation  are  forwarded  to  the 
official  grant  file. 

1 3 .  Page  14,  4th  bullet:  “For  the  period  of  our  review,  required  case  checklists  were 
not  prepared for  three  of  the  eight  demonstration  grants  that  expired  and  required 
closeout  Similarly,  neither  of  the  two  training  and  technical  assistance  grants 
had  the  required  case  checklists.  ” 

We  do  not  believe  that  this  statement  is  accurate.  As  explained  above,  as  part  of 
the  programmatic  and  fiscal  closure  process,  OJP  obtains  a  final  Financial  Status 
Report  and  required  programmatic  reports,  and  those  reports  and  other  closeout 
documentation  are  forwarded  to  the  official  grant  file.  Although  the  grant  period 
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may  have  ended  at  the  time  of  GAO’s  review,  the  programmatic  and  fiscal  closure 
may  not  have  been  completed. 

Now  on  p.  17. 

14. 

Page  15,  bullet:  " A  closeout  notification  and  grant  adjustment  notice  to  de- 
obligate  funds  or  advise  the  grantee  to  retain  records  for  3  years  was  not 
prepared for  four  of  the  eight  expired  demonstration  grants.  Similarly,  neither  of 
the  two  training  and  technical  assistance  grants  had  a  closeout  notification  and 
grant  adjustment  notice.  ” 

Although  the  grant  award  periods  had  expired,  the  closeout  notice  and  grant 
adjustment  notice  is  the  final  step  in  the  process.  When  GAO  conducted  their  file 
review  in  March  2001,  the  programmatic  and  fiscal  closure  process  may  not  have 
been  completed  for  the  grants  GAO  references  in  the  draft  report. 

Now  on  pp.  19-20. 

15. 

Page  1 7,  Recommendations  section,  1st paragraph:  ,(To  facilitate  and  improve  the 
management  of  program  grant  monitoring,  we  recommend  the  Attorney  General 
direct  OJJDP,  as  part  of  its  ongoing  review  of  monitoring  policies  and practices, 
to  determine  whether  the  monitoring  documentation  problems  that  we  identified 
were  an  indication  of grant  monitoring  requirements  not  being  met  or  of  a  failure 
to  document  activities  that  did,  in  fact,  take  place. " 

See  p.  12. 

Most  grant  managers  prepare  site  visit  reports  after  an  on-site  review  as  their 
travel  vouchers  are  not  forwarded  for  payment  unless  the  reports  are  attached. 

However,  the  grant  managers  do  not  routinely  forward  copies  of  the  reports  to  the 
official  grant  and  working  files.  A  copy  of  the  reports  are  filed  in  the  grant 
manager’s  travel  folders  by  a  division  secretary.  These  travel  files  were  not 
examined  by  GAO  during  the  review.  A  more  effective  recommendation  would 
be  that  OJJDP  move  forward  and  ensure  future  monitoring  is  appropriately 
documented  in  the  official  grant  files. 

Now  on  p.  20. 

16. 

Page  1 7,  Recommendations  section,  last  paragraph :  “Further,  grant  managers  ’ 
performance  workplans  should  include  clear  expectations  and  accountability  for 
monitoring  called  for  in  grant  managers  *  monitoring  plans  as  well  as  follow-up 
requirements  to  notify  and  assist  grantees  who  consistently  submit  late  or  no 
progress  or  financial  reports.  ” 

We  concur  with  this  recommendation.  For  your  information,  performance  work 
plans  put  into  effect  after  February  2001  for  grant  manager’s  in  the  OJJDP Js 

Special  Emphasis  Division  more  clearly  reflect  monitoring  responsibilities  and 
provide  for  better  supervisory  controls.  Additionally,  to  assist  Program  Managers 
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in  ensuring  that  files  are  in  order  and  that  documentation  of  monitoring  is  in  place, 
the  Special  Emphasis  Division  has  included  work  objectives  in  the  performance 
work  plan  for  one  staff  member  requiring  that  staff  member  to  conduct  file 
“audits”  as  directed  by  the  Special  Emphasis  Division  Director  or  Deputy 
Director. 

If  you  have  any  questions  or  need  additional  information  about  this  response,  please 
contact  me  on  (202)  307-5933  or  LeToya  Bryant,  OJP  Audit  Liaison,  on  (202)  514-0692. 
Thank  you  for  your  continued  cooperation  and  assistance. 


Mary  Lou  Leary 
Acting  Assistant  Attorney  G< 


leral 


cc:  Terrence  S.  Donahue 

Acting  Administrator 
Office  of  Juvenile  Justice  and 
Delinquency  Prevention 

Cynthia  J.  Schwimer 
Comptroller 

Office  of  Justice  Programs 

LeToya  Bryant 

Audit  Liaison 

Office  of  Justice  Programs 

OAAG  Executive  Secretariat 
Control  No.  20011872 
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Appendix  II:  Comments  From  the  Department 
of  Justice 


The  following  are  GAO’s  comments  on  the  Department  of  Justice’s 
September  19,  2001,  letter. 


GAO  Comment 


According  to  the  OJP  Handbook:  Policies  and  Procedures  for  the 
Administration  of  OJP  Grants  (Feb.  1992),  official  grant  files  kept  by 
the  Office  of  the  Comptroller  Control  Desk  are  to  contain  documents 
relating  to  each  grantee,  including  progress  and  financial  reports  and 
site  visit  reports.  In  addition,  for  documentation  to  be  readily  available 
for  examination,  as  required  by  the  Comptroller  General’s  internal 
control  standards,  keeping  them  in  the  official  grant  file  seems 
appropriate. 
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Appendix  III:  GAO  Contacts  and  Staff 
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GAO’s  Mission 

The  General  Accounting  Office,  the  investigative  arm  of  Congress,  exists  to 
support  Congress  in  meeting  its  constitutional  responsibilities  and  to  help 
improve  the  performance  and  accountability  of  the  federal  government  for  the 
American  people.  GAO  examines  the  use  of  public  funds;  evaluates  federal 
programs  and  policies;  and  provides  analyses,  recommendations,  and  other 
assistance  to  help  Congress  make  informed  oversight,  policy,  and  funding 
decisions.  GAO’s  commitment  to  good  government  is  reflected  in  its  core  values 
of  accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  is  through  the 
Internet.  GAO’s  Web  site  (www.gao.gov)  contains  abstracts  and  full-text  files  of 
current  reports  and  testimony  and  an  expanding  archive  of  older  products.  The 

Web  site  features  a  search  engine  to  help  you  locate  documents  using  key  words 
and  phrases.  You  can  print  these  documents  in  their  entirety,  including  charts  and 
other  graphics. 

Each  day,  GAO  issues  a  list  of  newly  released  reports,  testimony,  and 
correspondence.  GAO  posts  this  list,  known  as  “Today’s  Reports,”  on  its  Web  site 
daily.  The  list  contains  links  to  the  full-text  document  files.  To  have  GAO  E-mail 
this  list  to  you  every  afternoon,  go  to  our  home  page  and  complete  the  easy-to-use 
electronic  order  form  found  under  “To  Order  GAO  Products.” 

Order  by  Mail  or  Phone 

The  first  copy  of  each  printed  report  is  free.  Additional  copies  are  $2  each.  A 
check  or  money  order  should  be  made  out  to  the  Superintendent  of  Documents. 
GAO  also  accepts  VISA  and  Mastercard.  Orders  for  100  or  more  copies  mailed  to  a 
single  address  are  discounted  25  percent.  Orders  should  be  sent  to: 

U.S.  General  Accounting  Office 

P.O.  Box  37050 

Washington,  D.C.  20013 

To  order  by  Phone:  Voice:  (202)  512-6000 

TDD:  (301)  413-0006 

Fax:  (202)  258-4066 

Visit  GAO’s  Document 
Distribution  Center 

GAO  Building 

Room  1100,  700  4th  Street,  NW  (comer  of  4th  and  G  Streets,  NW) 

Washington,  D.C.  20013 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

■ - -  “ 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm, 

E-mail:  fraudnet@gao.gov,  or 

1-800424-5454  (automated  answering  system). 

Public  Affairs 

Jeff  Nelligan,  Managing  Director,  NelliganJ@gao.gov  (202)  5124800 

U.S.  General  Accounting  Office,  441  G.  Street  NW,  Room  7149, 

Washington,  D.C.  20548 
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